What is the Treadstone 71 Influence Operations Effectiveness Audit?
The Influence Operations Effectiveness Audit (IO-Audit) is an independent third-party verification of a counter-disinformation or counter-influence campaign's measurement against the ASA Measurement of Effect protocol (ASA-MOE, Version 1.0 draft for comment 0.1, DOI 10.5281/zenodo.23883472, https://doi.org/10.5281/zenodo.23883472). Source grades follow the ASA Evidence Standard, Public Edition 1.2, DOI 10.5281/zenodo.23137881. The auditor verifies four things: the design (pre-registration, hash, comparison design), the data (source grades on every input, instruments fielded as registered), the computation (the effect re-computed from raw data), and the Effect Statement (format, disclosures, falsifiers). The deliverable is a signed Audit Statement that sets the design grade DS-1 to DS-5, places the causal claim on a Sherman Kent band, and lifts or holds the self-measured cap of C on reliability. Treadstone 71 does not audit campaigns it designed, ran, or advised on. Three engagements: Design Audit before launch, $12,500 fixed, ten business days; Campaign Audit after the campaign, from $45,000, twenty business days from a complete data room; Program Audit for a funder's whole portfolio, quoted annually.
Influence Operations Effectiveness Audit
The auditor did not run the campaign. That is the product.
Counter-disinformation campaigns are measured, when they are measured at all, by the organization that ran them. The IO-Audit puts an independent party between the campaign and the claim: Treadstone 71 verifies the design, re-computes the effect from raw data, grades every input as a source, and signs a statement the funder can put in front of an oversight committee. All of it against the published ASA Measurement of Effect protocol, so the client knows the test before the test starts.
The audit at a glance
- Verifies: registration, comparison design, instruments, data, computation, and the Effect Statement.
- Delivers: a signed Audit Statement with the design grade, source grades, the re-computed effect, and the Kent band the evidence supports.
- Lifts: the self-measured cap of C on reliability, when the evidence earns it.
- Refuses: any campaign Treadstone 71 designed, ran, or advised on.
We do not grade our own campaigns.
Treadstone 71 will not audit a campaign it designed, ran, or advised on. Where the firm has done any of those, it names an independent auditor working to the same protocol and steps aside.
A measurement firm that also sells the campaign has one incentive. An auditor that sells only the audit has the opposite one.
Written into every engagement letter.
Four checks, one signature
Each check maps to a component of the ASA-MOE protocol, DOI 10.5281/zenodo.23883472. A campaign can pass three and fail one; the statement says which.
1 · The design
Was the design registered before the first message went out, and is the hash intact? Which of the five comparison designs was used, and was it applied as registered? The auditor sets the design grade, DS-1 to DS-5, from what was done, not what was planned.
2 · The data
Every input is a source and gets graded like one under the ASA Evidence Standard: the survey house, the platform data, the ministry's own indicator feeds. Were the instruments fielded as registered, blind to condition? Are response rates and weighting documented?
3 · The computation
The auditor re-computes the effect from raw data with the registered analysis plan. The re-computed number sits beside the reported one in the statement. A gap beyond rounding is a finding, and the statement says how large.
4 · The statement
Does the operator's Effect Statement conform to the format: registration reference, design grade, sample, effect with interval, band, falsifiers, disclosures, cost per unit of effect? Are the disclosures complete, including who paid, who ran, who measured?
The test is published before the audit starts
Every check runs against the criteria in the ASA Measurement of Effect Protocol (ASA-MOE), Version 1.0, draft for comment 0.1, deposited on Zenodo under DOI 10.5281/zenodo.23883472. Source grades on every data input follow the ASA Evidence Standard, Public Edition 1.2, DOI 10.5281/zenodo.23137881: reliability A to F and credibility 1 to 6, printed apart, never averaged.
| Check | Verification criterion | ASA-MOE reference |
|---|---|---|
| Design | Registration timestamped and hashed before launch, hash intact. Comparison design applied as registered. Design grade DS-1 to DS-5 set from what was done. | Components 1 and 2 |
| Data | Instruments fielded as registered, blind to condition. Response rates and weighting documented. Every input graded as a source. | Components 3 and 4; ASA Evidence Standard |
| Computation | Effect re-computed from raw data with the registered analysis plan. A gap beyond rounding is a finding. | Components 1 and 5 |
| Statement | Every Effect Statement field present, with disclosures, falsifiers, and cost per unit of effect. Kent band at or under the ceiling the design grade sets. | Components 5 and 6 |
| Determination | MOE-Audited granted only when design, data, and computation verify. Otherwise the self-measured cap of C on reliability holds. | Component 5; MOE-Audited mark |
Three ways to bring in an auditor
Prices are fixed and published so a funder can budget the audit into the grant. Every engagement runs under mutual non-disclosure.
Catches the measurement failures while they can still be fixed, and costs a fraction of a failed campaign.
- Review of the registration against the protocol
- Instrument and indicator check
- Comparison design assessment and DS grade forecast
- Minimum detectable effect and sample check
- Written findings within ten business days
- Earns the MOE-Registered mark on the auditor's word
The full verification. Price rises with the number of populations, instruments, and indicator feeds.
- Everything in the Design Audit, applied retrospectively
- Re-computation of the effect from raw data
- Source grades on every data input
- Deviation and exposure log review; design grade set
- Findings graded minor, major, critical
- Signed Audit Statement twenty business days from a complete data room
- Earns the MOE-Audited mark; the cap lifts
For a funder or ministry with a portfolio of campaigns and vendors that has to report on all of it.
- Standing audit of every campaign in the program
- Cross-vendor comparison on cost per unit of effect
- Protocol conformance clause for vendor contracts
- Design Audits on every new campaign as standard
- Annual program Effect Statement for oversight
- Named auditor on call
Sector pricing. Government, defense prime contractors, and academic institutions reduce cost by 15 percent, the same terms as The Adversary Index. Multi-campaign programs are quoted on the number of campaigns and vendors.
Five steps, twenty business days
The clock starts when the data room is complete. The checklist for the data room ships with the engagement letter, so there is no ambiguity about when that is.
Engagement and conflict check
Treadstone 71 confirms it had no role in the campaign, signs the mutual NDA, and issues the data-room checklist.
Day 0Data room
The client assembles the registration, instruments, raw data, logs, and cost. The auditor confirms completeness or lists the gaps.
Clock starts when completeVerification and re-computation
Design check, source grades, re-computation from raw data with the registered plan, deviation review.
Days 1 to 12Draft findings
The client receives the findings and has five business days to supply corrections or context. Facts can change a finding; argument cannot.
Days 13 to 17Signed Audit Statement
Final statement issued under the auditor's name, with the design grade, the re-computed effect, the band, and the determination on the MOE-Audited mark.
Day 20
| The client provides | Why the auditor needs it |
|---|---|
| Registration file, timestamp, hash | Proves the design was fixed before launch and has not changed since. |
| Instruments as fielded | Checked against the registered instrument, item by item. |
| Raw survey data, sampling and weighting notes | The effect is re-computed from here, not from the operator's tables. |
| Indicator feeds with attribution windows | Behavioral indicators are graded as sources and checked against their registered windows. |
| Exposure log and deviation log | Who was reached, when, and every departure from the registered plan. |
| Campaign cost | For the cost-per-unit-of-effect line in the statement. |
| The operator's Effect Statement or report | The claim under audit, verified line by line. |
| Contacts at the survey house and data providers | So each can be graded as a source on its own record. |
What the client receives
Illustrative. The campaign, parties, and numbers are synthetic, built to show the format. It audits the same synthetic campaign the ASA-MOE page uses as its worked example.
Verification results
| Component | Result | Note |
|---|---|---|
| Registration before launch | Verified | Hash matches the timestamped file; no post-launch edits. |
| Comparison design | Verified | Stepped-wedge across four regions, order drawn at random as registered. Design grade set: DS-2. |
| Instrument as fielded | Verified | Five items, identical pre and post; fielded blind to region status. |
| Data inputs | Graded | Survey house Z B2; platform search data B2; ministry information-line counts C2, interested source. |
| Re-computation | Matched | Belief-scale effect re-computed at −6.3 against −6.4 reported; within rounding. |
| Deviation log | One finding | See Finding 1. |
| Effect Statement format | Conforms | All fields present; disclosures complete. |
Findings
| # | Severity | Finding | Effect on the claim |
|---|---|---|---|
| 1 | Minor | The attribution window for the information-line indicator was extended from 7 to 10 days after launch. Logged by the operator on 2027-04-11. | The indicator is reported, not weighted; no change to the band. |
| 2 | Minor | Response-rate reporting omitted the per-region breakdown. | Supplied during the draft-findings window; statement updated. |
Re-computed effect
Determination
Design grade DS-2. Data inputs graded B2, B2, C2. The registered hypothesis, a reduction of at least 5 points, is supported. Causal judgment the evidence supports: Highly likely (85 to <95) the campaign caused a reduction in belief of at least 5 points, the ceiling for DS-2; source grades do not lower it. The eight-week persistence claim remains Likely (55 to <85) pending the registered second wave.
A statement that finds a critical deviation, an unregistered analysis, or a re-computation gap outside rounding says so in the same format, and the cap stays.
Anyone who has to answer for the spend
Prove it to the committee
A signed statement from a party with no stake in the result answers the question a parliamentary committee, an inspector general, or a board audit chair will ask.
Compare vendors on one rubric
When three vendors each report their own success, cost per unit of audited effect is the only number that compares them.
De-risk the subcontract
A Design Audit before launch and a Campaign Audit before final payment turn a measurement clause into a verified deliverable.
Get the cap lifted
An operator that wants its result to grade above C on reliability commissions the audit itself, or writes it into the proposal. The protocol tells the operator what the auditor will look for; nothing is hidden.
Independence, in four lines. Treadstone 71 does not audit campaigns it designed, ran, or advised on, and names an independent auditor where it has. The audit is sold on its own; no campaign work is bundled with it. Engagements run under mutual NDA and the firm discloses nothing about the client, the vendor, or the result. The Audit Statement is the client's document, published or not at the client's choice, under the auditor's name either way.
Before you commission one
What does an IO-Audit verify?
Four things, against the ASA-MOE protocol. The design: was the registration made before launch, is the hash intact, and which of the five comparison designs was used. The data: does every input carry a source grade, and were the instruments fielded as registered. The computation: does the effect re-compute from raw data to the number the operator reported. The statement: does the Effect Statement conform to the format, with disclosures and falsifiers. The auditor sets the design grade, DS-1 to DS-5, and signs.
Why can the campaign operator not do this itself?
It can, and the protocol tells it how. What it cannot do is lift the cap. Under ASA Rule 1, a self-serving claim grades no higher than C on reliability, and a campaign measured only by the organization that ran it is a self-serving claim about its own work. The cap lifts when an independent party with no role in the campaign verifies the design, the data, and the computation. That is the audit.
Does Treadstone 71 audit its own work?
No. Treadstone 71 will not audit a campaign it designed, ran, or advised on. Where the firm has done any of those, it names an independent auditor working to the same protocol and steps aside. The rule is written into every engagement letter.
What does the client have to provide?
The registration file with its timestamp and hash; the instruments as fielded; raw survey data with sampling and weighting documentation; indicator feeds with their attribution windows; the exposure log; the deviation log; campaign cost; the operator's own Effect Statement or equivalent report; and contact details for the survey house and platform-data providers so the auditor can grade them as sources. A data-room checklist ships with the engagement letter.
What comes out at the end?
A signed Audit Statement of two to four pages: what was verified, what was not, findings graded by severity, the design grade set, source grades on every input, the re-computed effect beside the reported one, the Sherman Kent band the evidence supports, and the auditor's determination on the MOE-Audited mark. The statement is the client's to publish, under the auditor's name.
How long does it take, and what does it cost?
A Design Audit, before launch, is $12,500 fixed and returns written findings within ten business days. A Campaign Audit, after the campaign, starts at $45,000 for a single campaign and returns the signed Audit Statement twenty business days after the data room is complete. A Program Audit covers every campaign in a program for a year and is quoted on the number of campaigns and vendors.
Is the audit confidential?
Yes. Engagements run under mutual non-disclosure. Treadstone 71 does not disclose the client, the campaign, the vendor, or the result. The Audit Statement is the client's document; publication is the client's choice. The auditor's name stays on it either way.
Before launch is cheaper. After is still worth it.
Send the campaign's stage, the number of populations and vendors, and the reporting deadline. Treadstone 71 replies with the conflict check, the engagement letter, and the data-room checklist.
Direct line: 424.234.3629
